Video Scene Segmentation using Video and Audio Features
نویسندگان
چکیده
In this paper we present a novel algorithm for video scene segmentation. We model a scene as a semantically consistent chunk of audio-visual data. Central to the segmentation framework is the idea of a finite-memory model. We separately segment the audio and video data into scenes, using data in the memory. The audio segmentation algorithm determines the correlations amongst the envelopes of audio features. The video segmentation algorithm determines the correlations amongst shot key-frames. The scene boundaries in both cases are determined using local correlation minima. Then, we fuse the resulting segments using a nearest neighbor algorithm that is further refined using a time-alignment distribution derived from the ground truth. The algorithm was tested on a difficult data set; the first hour of a commercial film with good results. It achieves a scene segmentation accuracy of 84%.
منابع مشابه
Traffic Scene Analysis using Hierarchical Sparse Topical Coding
Analyzing motion patterns in traffic videos can be exploited directly to generate high-level descriptions of the video contents. Such descriptions may further be employed in different traffic applications such as traffic phase detection and abnormal event detection. One of the most recent and successful unsupervised methods for complex traffic scene analysis is based on topic models. In this pa...
متن کاملCompressed Domain Scene Change Detection Based on Transform Units Distribution in High Efficiency Video Coding Standard
Scene change detection plays an important role in a number of video applications, including video indexing, searching, browsing, semantic features extraction, and, in general, pre-processing and post-processing operations. Several scene change detection methods have been proposed in different coding standards. Most of them use fixed thresholds for the similarity metrics to determine if there wa...
متن کاملVideo Scene Segmentation System Using Audio-visual Features
This work demonstrates a new approach to video temporal segmentation into scenes. The utilized technique is based on an audio-visual extension of the well-known method of the Scene Transition Graph (STG). This multi-modal extension exploits both lowand high-level audio-visual descriptors to construct distinct STGs. These STGs are employed into a probabilistic framework that is used for estimati...
متن کاملAudio-Assisted Scene Segmentation for Story Browsing
Content-based video retrieval requires an effective scene segmentation technique to divide a long video file into meaningful high-level aggregates of shots called scenes. Each scene is part of a story. Browsing these scenes unfolds the entire story of a film. In this paper, we first investigate recent scene segmentation techniques that belong to the visual-audio alignment approach. This approac...
متن کاملVideo Segmentation with the Support of Audio Segmentation and Classification
Video structure extraction is essential to automatic and contentbased organization, retrieval and browsing of video. However, while many robust shot segmentation algorithms have developed, it is still difficult to extract scene structures or group shots into scenes. In this paper, we present a novel audio assisted video segmentation scheme, in which audio and color information is integrated in ...
متن کامل